(1) Field of the Invention
The present invention relates to a three-dimensional image processing apparatus which generates a two-dimensional image with reality from the three-dimensional information set as the numerical data in the apparatus.
(2) Description of the Prior Art
Recently, the computer graphics have been drawing attention as the backbone technology for the virtual reality, multi-media and the like. In the computer graphics, a polyhedral model obtained by approximating a complicated surface of an object with small polygons is utilized. A natural three-dimensional image close to the reality is then generated by attaching a texture on the surface thereof. There are various types of the image processing apparatus for generating such a three-dimensional image, which will be described in detail below.
[First conventional example]
FIG. 1 is a block diagram showing the first conventional example of the image processing apparatus. This apparatus comprises a main memory 100 for storing the polygon data and the texture data, a CPU 101, a texture memory 102 for storing the pattern data on the polygon surface, a polygon mapping processing device 103, a frame memory 104 for one screen, a display controller 105, and a display 106, and the content of the frame memory 104 is read out according to the screen coordinates value from the display controller 105, and being output in sequence to the display 106.
With this processing apparatus, the CPU 101 executes the modeling transformation, projective transformation and the like using a parameter defining the polygon stored in the main memory 100, and an operation data obtained according to the application software and the operator's operation, to transform the parameter of the corresponding polygon so as to form a polygon image, and outputs the parameter to the polygon mapping processing device 103 in sequence.
The polygon mapping processing device 103 takes out the texture data corresponding to the polygon among the input parameters of polygon from the texture memory 102, and while attaching the texture data to each polygon, subjects each polygon to the blinding processing, and outputs the results to the frame memory 104. When the texture is not stored in the texture memory 102, the corresponding texture is read out from the main memory 100, under the instruction of the CPU 101. The polygon image generated by the polygon mapping processing device 103 is written in the frame memory 104 as the frame picture, and repeatedly read out onto the display 106 at a required speed, and displayed as a static image.
Moreover, as shown in FIG. 2, there is an image processing apparatus disclosed in Japanese Patent Application Laid-Open Hei 7 No. 254072. In this apparatus, the polygon mapping processing device 103 is divided into a coordinates transformation device 107 and a rendering device 108. The coordinates transformation device 107 performs the processing up to the projection of a model from the three-dimensional coordinates space to the two-dimensional coordinates space, and the rendering device 108 generates the screen image, while attaching the texture data to each polygon.
[Second conventional example]
FIG. 3 is a block diagram showing the second conventional example of the image processing apparatus. This image processing apparatus generates the image which looks sharp to observers at a high speed, by making the resolution of the image high in the vicinity of a marked point, and low in a place away from the marked point. This is disclosed in Japanese Patent Application Laid-Open Hei 5 No. 6402.
This image processing apparatus comprises marked point-detecting means 110 for detecting the marked point of the observer to output the information of the marked point, a pixel selector 111 for selecting the pixel more densely as it approaches the marked point based on the marked-point information being output by the marked point-detecting means 110, and selecting the pixel more roughly as it is away from the marked point, a luminance calculator 112 for calculating and accurately determining the luminance value of the pixel selected by the pixel selector 111, an interpolator 113 for determining the luminance value of the uncalculated pixel by the interpolation using the luminance value determined by the luminance calculator 112, and display means 114 for displaying the generated image.
With this processing apparatus, first the marked point-detecting means 110 detects the marked point of the observer, and outputs the information of the detected marked point to the pixel selector 111. The pixel selector 111, based on the information of marked point received from the marked point-detecting means 110, selects the pixel more densely as it approaches the marked point, while selects the pixel more roughly as it is away from the marked point, and outputs the coordinates of the selected pixel to the luminance calculator 112. The luminance calculator 112 calculates and accurately determines the luminance value of the pixel on the coordinates being output from the pixel selector 111, and outputs the luminance value together with the coordinates of the pixel whose luminance value has been determined to the interpolator 113. The interpolator 113 determines the luminance value of the uncalculated pixel by means of the interpolation using the luminance value determined by the luminance calculator 112, and outputs the determined luminance values of the whole pixels to the display means 114. The display means 114 displays the image based on the luminance values of the whole pixels received from the interpolator 113. This operation is repeated until the values of the whole pixels in the image are determined.
[Third conventional example]
FIG. 4 is a block diagram showing the third conventional image processing apparatus. This image processing apparatus is to reduce the pixel-generating processing, which was proposed by the present applicant.
This image processing apparatus comprises an modeling data memory 120 for storing the model data, a CPU 121 for controlling the whole apparatus, a coordinates transformation device 122 for transforming the coordinates of the vertex of a polygon and transmitting the vertex coordinates after the coordinates transformation and an identifier of the polygon (hereinafter referred to as "polygon code") to a polygon code memory, the polygon code memory 123 for storing the polygon code in a position to draw the polygon, a polygon cache 124 for receiving the vertex coordinates after the coordinates transformation and a rendering parameter (such as a texture coordinates) and storing them in an address indicated by the polygon code, an internally divided value generator 125 for generating an internally divided value from the vertex coordinates and the screen coordinates, a rendering device 126 for receiving the internally divided value and the rendering parameter to generate color data, a display controller 127 for outputting the image signal according to the color data value, and a display 128 for displaying the image based on the image signal.
With this method, not the color data value but the polygon code is written in the polygon code memory 123. Furthermore, synchronously with the horizontal and vertical signals of the display 128, pixel processing such as mapping and rendering is performed. Therefore, the processing becomes very light.
[Fourth conventional example]
Furthermore, as disclosed in Japanese Patent Application Laid-Open Hei 5 No. 290145 and Japanese Patent Application Laid-Open Hei 8 No. 87585, there can be considered a method for reducing the amount to be processed by utilizing the fact that when the observer who watches the virtual space and a model in the virtual space move relatively, even if the model is displayed with rougher accuracy as the relative speed increases, it cannot be perceived by a human being, to reduce the number of vertexes (or number of polygons).
FIG. 5 is a block diagram showing the fourth conventional example of the three-dimensional image processing apparatus.
This apparatus comprises an image information input unit 130, an object-detecting section 131, objects 132-1.about.132-n, sections for detecting the speed of the object 133-1 .about.133-n, a visual point position-detecting unit 134, a region-dividing section 136, blocks 137-1.about.137-n, and a synthesizing display unit 138.
The object-detecting section 131 detects the image being input by the image information input unit 130, and the speed detecting section 133 detects the speed of the object 132 in the image. Depending upon the position of the visual point, the relative speed-detecting section 135 detects the relative speed. The region-dividing section 136 divides the object into a plurality of regions, and changes the number of vertexes of each region depending upon the detected relative speed, to synthesize the respective model images and display them on the synthesizing display unit 138. Thus, with increase of the relative speed, a model with rougher accuracy is displayed.
In order to generate a computer graphic picture having a high quality with reality, it is necessary to arrange multiple shape data in the virtual space, and to prepare lots of textures having the high resolution which constitute the surface of the shape. Therefore, since a large volume of data is processed, a large volume of processing has to be performed to draw one screen. In general, such a large volume of data is stored in the main memory, and read out according to need. However, when a observer moves, for example, pans, it is necessary to form the image following this moving speed. In this case, the volume of data to be processed becomes enormous, causing problems described below.
The first conventional example performs the display processing for each polygon, hence in the case of an image that, for example, a book is placed on a desk, processing of the desk and processing of the book are performed, and processing such as deleting the image of the desk in the overlapped portion is performed. Therefore, painting of screens is superimposed frequently, causing a problem that the processing for forming the pixel becomes heavy. Accordingly, when the observer himself/herself moves, the processing for generating the image becomes slow, and the image cannot follow the movement of the observer, thus forming the factitious image.
The second conventional example is capable of speeding up the image-generating processing compared to the speed of the first conventional example by decreasing the resolution of the image at a place away from the point marked by the observer. However, the portion of the marked point requires the conventional image processing, and when the head position of the observer moves, the same processing as that of at the time of standing still is performed at the time of moving of the head position, in the marked point. Therefore, the image is behind with respect to the movement of the head position. Moreover, the number of drawing per 1 second is restricted, thus a smooth movement cannot be obtained.
The third conventional example has light processing for writing the polygon code, not the data value of the color, in the polygon code memory, and performs pixel processing such as mapping and rendering, synchronously with the horizontal and vertical signals of the display. Therefore, superimposing of painting or the like is not caused and the pixel processing becomes very light, compared to that of the first conventional example. However, with this conventional example, the image data itself to be processed is almost the same as that of the first conventional example, thereby there may be a case where the image is behind with respect to the movement of the observer, and the movement is not smooth.
With the fourth conventional example, the processing volume of the data can be reduced, by displaying a model with rougher accuracy. as the relative speed of the observer and the model in the virtual space increases. However, even if it is a model having a simplified rough accuracy, if a conventional texture is used as the texture to be attached thereon, the data volume of the texture becomes quite large. Therefore, there may be a case where the image cannot follow the movement of the observer.
As described above, according to the conventional apparatus, even when the observer moves around, the screen is generated and displayed with the same quality as that of when the observer stands still. Furthermore, drawing processing is frequently repeated which is necessary for the position of the observer who newly moved. Accordingly, the volume of processing increases, and the drawing processing cannot catch the movement of the head position, thus the drawing position is behind with respect to the movement of the head position.
Furthermore, in order to make a virtual space with reality, a texture having the high resolution is often used. However, in order to hold all the textures in the virtual space in the texture memory in the graphic apparatus, the capacity of the texture memory becomes quite large, which is not realistic. Moreover, the texture must be read in every time when the texture which is not present in the texture memory is required for the drawing. However, a lot of time is required for reading the texture having the high resolution at a time, which makes it difficult to generate an image at a high speed, thus the drawing position is behind with respect to the movement of the head position.